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Section 12 



Levy's Continuity Theorem. Poisson 
Approximation. Conditional 
Expectation. 



Let us start with the following bound. 

Lemma 27 Let X be a real-valued r.v. with distribution P and let 



Then, 



Proof. Since 



we have 



f(t) =Ee ltx = J e Ux d¥(x). 
i\X\>-)<- [ U (l-Ref(t))dt. 



Re/(t) = J cos txdF(x) 

U U 

-J J (I - cos tx)dF(x)dt = -J J(l-costx)dtdP(x) 



u 

OR R 



1 )dP(x) 

xu J 



1 )dP(x) 

\ xu J 

\xu\>\ 

{since ^ < ^ if y> l} > (1-sinl) J ldF{x) > *p(|X| > 



|xu|>l 



Theorem 28 (Levy continuity) Let (X n ) be a sequence of r.v. on R k . Suppose that 

f n (t) = EeW^ - f(t) 



46 



and f(t) is continuous at along each axis. Then there exists a probability distribution P such that 

f(t) = J e l{t ^dV{x) 

and C(X n ) -> P. 

Proof. By Lemma 19 we only need to show that {£(!„)} is uniformly tight. If we denote 

X n = (X n .i, . . . , X n ^) 

then the c.f.s along the coordinate: 

/* (ti) := /„(0, ..../,.()....(). Ee"' x »-' -> /(0, ...,^,...0) =: f (U). 
Since /„(0) = 1 and, therefore, /(0) = 1, for any e > we can find <5 > such that for alH < k 

\f(U)~l\<e if \U\<5. 

This implies that for large enough n 

\f n (U)-l\<2s if \U\<8. 

Using previous Lemma, 

P(|*n,i| >^)<lj*(l- Ref^dU <\[\ l - fn{U)\dU < 7 ■ 2e. 
The union bound implies that 

¥(\X n \ > ^) < Uke 

and {£(X n )} n >i is uniformly tight. 

□ 

CLT describes how sums of independent r.v.s are approximated by normal distribution. We will now give a 
simple example of a different approximation. Consider independent Bernoulli random variables X™ ~ B(pf) 
for i < n, i.e. P(X™ = 1) = pf and ¥{Xf = 0) = 1 - pf . If p? = p > then by CLT 

Sn - UP Af(0,l). 



y/np(l-p) 

However, if p = p" — > fast enough then, for example, the Lindeberg conditions will be violated. It is 
well-known that if pf = p n and np n — > A then S n has approximately Poisson distribution LTa with p.f. 

/(fc) = ^e- A for fc = 0,l,2,... 

Here is a version of this result. 

Theorem 29 Consider independent Xi <~ for i < n and let 

S n = Xi + . . . + X n and A = pi + . . . + p n . 
Then for any subset of integers BCZ, 

|P(5„eB)-n A (B)| <J> 2 . 
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Proof. The proof is based on the construction on "one probability space". Let us construct Bernoulli r.v. 
Xi ~ B(pi) and Poisson r.v. X* ~ H Pi on the same probability space as follows. Let us consider a probability 
space ([0, 1],B, A) with Lebesque measure A. Define 



Xi = Xi(x) = 



0, 0<x<l- Pi , 

1, 1 — Pi < x < 1 . 



Clearly Xi ~ B(pi). Let us construct X* as follows. If for fc > we define 



0<Z<fe 



then 



X, = ^(z) = { 



0, < x < c , 

1, Co < x < ci, 

2, Ci < x < c 2 , 



Clearly X* ~ H Pi . When Xi ^ X*l Since 1 — pj < e Pj — c , this can only happen for 

1 — Pi < x < c and C\ < x < 1, 



i.e. 



P(^- X*) = & (1 - Pj ) + (1 - e-w - p . e -«) = Pj (l e -*>) < p] 



We construct pairs (Xi, X*) on separate coordinates of a product space, thus, making them independent fo 
i < n. It is well-known that ^2 i<n X* ~ LTa and, finally, we get 

p(s n ^s;) < E p (*> ^ ^ E^ 2 - 

j<n j<n 

□ 

Conditional expectation. Let (f2,£>, P) be a probability space and X : £1 — > R be a random variable such 
that E\X\ < oo. Let ^4 be a a-subalgebra of B, AC. B. 

Definition. Y = E(X\A) is called conditional expectation of X given A if 
1. Y : fl — > K is measurable on .4, i.e. if S is a Borel set on R then Y _1 (.B) e .4. 



2. For any set A e .4 we have EXI^ = EYIa, where /^M = { q ui 



e A 
iA. 



Definition. If X, Z are random variables then conditional expectation of X given Z is defined by 

Y = E(X\Z) = E(X\a(Z)). 

Since Y is measurable on <r(Z), Y — f(Z) for some measurable function /. 
Properties of conditional expectation. 

1. (Existence of conditional expectation.) Let us define 



KA) = I 

J A 



XdP for Ae A. 

A 



n(A) is a (7-additive signed measure on A. Since X is integrable, if F(A) = then n(A) — which means 
that /x is absolutely continuous w.r.t. P. By Radon-Nikodym theorem, there exists Y = ^ measurable on A 
such that for A e A 

fi(A) = [ XdP = [ YdP. 

J A J A 
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By definition Y = E(X\A). 

2. (Uniqueness) Suppose there exists Y 1 = E(X\A) such that P(Y ^ Y 1 ) > 0, i.e. 

P(Y > Y') > or P(Y < y') > 0. 

Since both Y, Y' are measurable on A the set A = {Y > Y'} e A. One one hand, E(Y - Y')\ A > 0. On the 
other hand, 

E(Y - Y') 1 ^ = e ^!a - EXI A = 

- a contradiction. 

3. E(cX + Y\A) = cE(X\A) + E(Y\A). 

4. If cr-algebras C C A C B then 

E(E(X|^)|C) = Epf|C). 

Consider a set C £ C C i. Then 

EI C (E(E(X|.4)|C)) = EI C E(X|.4) = EI r A' and EI C (E(X|C)) = EXI C . 

We conclude by uniqueness. 

5. E(X\B) = X, E(X\{$, Q}) = EX, E(X\A) = EX if X is independent of A. 

6. If X < Z then E(X|*4) < E(Z|„4) a.s.; proof is similar to proof of uniqueness. 

7. (Monotone convergence) If E\X n \ < oo,E\X\ < oo and X n | X then E(X n \A) T E(X|.4). Since 

E(X n \A) < E(X n+1 \A) < E(X\A) 

there exists a limit 

3 = lim E(X n \A) <E(X\A). 

n — >oo 

Since E(X„|_4) are measurable on A, so is g = limE(X n \A). It remains to check that 

for any set A € A, EgI A = EXl A . 
Since X n l A f Xl A and Epf n |.4)l4 f #1^, by monotone convergence theorem, 

EX n l A t EXU and El A E(X n \A) T Ec?I A . 

But since EI j4 E(X„|y4.) = EX„Ia this implies that EgI A = EXI A and, therefore, g = E(X\A) a.s. 

8. (Dominated convergence) If \X n \ < Y, EY < oo, and X n — > X then 

limE(X„|A) = E(X\A). 

We can write, 

-Y < g n = inf X m < X n < h n = sup X m < Y. 

m>n m>n 

Since 

g n ]X, h n [X, \g n \ < Y, \h n \ < Y 

by monotone convergence 

E(g n \A) T E(X\A), E(h n \A) i E(X\A) => EX n \A) -» E(X|A). 

9. If E|X| < oo,E|XY| < oo and Y is measurable on A then 

E(XY|.4) = YE(X|.4). 
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We can assume that X, Y > by decomposing X = X + — X , Y = Y + — Y . Consider a sequence of simple 
functions 

Y n = Y^ w k l c k , C k eA 

measurable on A such that < Y n t Y. By monotone convergence theorem, it is enough to prove that 

E(Xl Ck \A)=l Ck E(X\A). 

Take B e A. Since BC k e A, 

EI B I Ck E(X\A) = EI BCk E(X\A) = EXI BCk = E(XI Ck )I B . 
10. (Jensen's inequality) If / : R — > E is convex then 

f(E(X\A)) <E(f(X\A)). 

By convexity, 

f(X) - f(E(X\A)) > df(E(X\A))(X - E(X\A)). 
Taking condition expectation of both sides, 

E(f(X)\A) - f(E(X\A)) > df(E(X\A))(E(X\A) - E(X\A)) = 0. 

□ 
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